Random walk in a finite directed graph subject to a 

road coloring 



Kouji YanSW^ 



August 3, 2010 



Abstract 



o 

(N 



A necessary and sufficient condition for a random walk in a finite directed graph 
subject to a road coloring to be measurable with respect to the driving random road 
colors is proved to be that the road coloring is synchronizing. For this, the random 
walk subject to a non-synchronizing road coloring is proved to have uniform law on 
a certain partition of the state space. 
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1 Introduction 

Let us consider a finite directed graph of constant outdegree. See, for example, Figure 
1 below; there are five sites and from each site there are two oneway roads laid. Let us 
color every road blue or red so that no two roads running from the same site have the 
same color. See, for example. Figure 2 below; the thick roads are colored red and the thin 
ones blue. 





Figure 1. Figure 2. 

We call A^ a random color if A^ is a random variable which takes values in the set of 
road colors; in this case, red and blue. To put it roughly, we mean by random walk a pair 
of processes {X, N} where A^ = {Nk)k&z is a sequence of colors which are independent 
and identically distributed (abbreviated as IID) and X = {Xk)^^^ is a site-valued process 
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which moves at each step from X^^i to X^ being driven by the random color A^^. We 
have the following table for instance: 
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In this table, we let /cq be a certain time and we assume that at the 5 steps to the time 
k = ko the random colors are blue, red, blue, blue and red, in this order. Then, no matter 
how the process X moves before ko — 5, the value of Xkg falls 2, and therefore the values 
of X afterward can be known from the values of N. We are interested in the necessary 
and sufficient condition that we can always know the values of X only from the values of 
N. 



1.1 Random walk subject to a road coloring 



Let V he a set of finite symbols. A matrix [A{y , x)]y^xev whose entries are non-negative 
integers is called an adjacency matrix and the pair (V, A) a directed graph. Note that there 
may be multiple edges which are not distinguished from each other. We call each element 
of V" a site. For each y,x & V, the value of A{y,x) may be regarded as the number of 
(oneway) roads from x to y. (We prefer to write A{y,x) than write A{x,y).) 

From each site x &V there are as many roads as deg(a;) := X]«gv^(^'^)- '^^^ graph 
{V, A) or the adjacency matrix A is called d-out if deg(x) = d\ in other words, there are 
d roads from each site. It is called of constant outdegree if it is d-ovX for some d. Write 



S = the set of all mappings from V to itself. 



:i.i) 



For (Ti, 0-2 G S and x G V, we write a20'ix simply for a2{o'i{x)). The semigroup S acts on 
V as follows: 



{cFia2)x = ai{a2x), ai,a2 G S, x e V. 



;i.2) 



Each element a G S may be identified with the 1-out adjacency matrix [(T{y,x)]y^xev via 
equation 



a{y,x) 



L{y=ax}- 



An adjacency matrix A of constant outdegree admits a family C = {a*-^-*, 
(possibly with repeated elements) such that 



(1.3) 
a^'^)} of S 

(1.4) 



Such a family C will be called a road coloring, because C indicates one of the ways of 
coloring the d roads running from each site so that no two roads from the same site have 
the same color. 

Let /i be a probability law on S. We write Supp(/i) for the support of /i, i.e., 

Supp(/i) = {a G S : ii{a) > 0} . (1.5) 

Enumerating Supp(/i) as {a^^\ . . . ,0"'^'^^}, we define the adjacency matrix A by fll.4p so 
that Supp(/i) is a road coloring of (V, A). The resulting directed graph (V, A) is called the 
directed graph induced by [i. Now we introduce random walk in a directed graph indexed 
by Z :={... , —1, 0, 1, . . .} as follows. 

Definition 1.1. Let // be a probability law on S. A pair of processes {X,N} defined 
on a probability space is called a ^-random walk if X = (Xfc)^^^ and A^ = {Nk)k£Z are 
processes taking values in V and S, respectively, such that the following statements hold: 

(i) Nk is independent of ct(Xj, Nj : j < k — 1) for each A; G Z; 
(ii) A^ = {Nk)k^i is IID with common law yu; 
(iii) it holds that 

Xk = NkXk-u a.s., keZ. (1.6) 

Let (V, A) denote the directed graph induced by fi. The process X = {Xk)kez moves 
at each step from a site to another in {V,A), being driven by the randomly-chosen road 
colors indicated by A^ = (A'fc)fcgz via equation (II. 6p . This is why we call such a process 
{X, N} a random walk. Note that our definition is different from the one in a lot of 
literatures; see [13] and references therein. 



Definition 1.2. A /i-random walk {X,N} is called strong if Xk is a.s. measurable with 
respect to cr(Aj- : j < k) for all A; G Z; or equivalently, there exist measurable mappings 
/fc : S^ ^ y such that 

Xk = fkiNk, Nk-i, . . .) a.s. for all A; G Z. (1.7) 

The purpose of this paper is to investigate a necessary and sufficient condition for the 
/i-random walk to be strong. 



1.2 Main theorem 

Let A^ denote the identity matrix and define v4"(?/, x) = J2z€V ^^'^iv^ z)A{z, x), y,x E V 
recursively for n > 1. A directed graph {V,A) is called strongly- connected if for any 
y,x E V there exists n = n{y,x) > 1 such that A^{y,x) > 1; or in other words, one 
can walk from every site to every other site. The graph (V, A) is called aperiodic if the 



period at x & V, i.e., the greatest common divisor among {n > 1 : A'^(x,x) > 1}, is one 
for all X & V. Note that {V, A) is both strongly connected and aperiodic if and only if 
there exists a positive integer r such that A'"(?/,x) > 1 for all y,x &V. We say that the 
directed graph (V, A) satisfies the assumption (A) if it is of constant outdegree, strongly- 
connected, and aperiodic. We will prove as Theorem 12.81 that under the assumption (A) 
there exists a unique /x-random walk {X, N} which is stationary. 

For (T G S, we write aV = {ax : x G V}. A road coloring C is called synchronizing if 
there exists a sequence s = (up, . . . , cti) of road colors such that the composition (s) : = 
(Tp- ■ -ai maps V onto a singleton; or in other words, those who walk in the directed graph 
being driven by the road colors ai, . . . , cXp in this order will arrive at a common site. 

Now one of our main results is as follows. 

Theorem 1.3. Let fi be a probability law on S and let {X,N} be a fi-random walk. 
Suppose that the directed graph induced by /i satisfies the assumption (A). Then the 
following three assertions are equivalent: 

(i) Supp(/x) is synchronizing. 

(ii) The limit lim NkN^-i ■ ■ ■ A^j+i exists a.s. for all k E Z. 

(iii) The ^-random walk {X, N} is strong. 

Theorem II. 31 will be proved in Section |H Note that the most difficult part of Theorem 
11.31 is to show that (iii) implies (i). We shall prove the contraposition: 

If Supp(/i) is non-synchronizing, then the /i-random walk {X, X} is non-strong. (1.8) 



1.3 A typical sufficient condition 

To prove 01.81) . we need to find some extra randomness which is not measurable with 
respect to (j{Nj : j < k). The key to the proof is to reveal a certain symmetry, or to put 
it more precisely, to construct another random walk from the original random walk by 
stopping it at certain stopping times and the problem is then reduced to the proposition 
given as follows. 

Let &iV) denote the permutation group of V, which may be regarded as a subgroup 
of S. 

Proposition 1.4. Let {X,N} be a fi-random walk. Suppose that Supp(/i) is contained 
in &{V) and that 

/x-(^eS:a(z)=j) ^-i-, t^jeV (1.9) 

n-!>oo ]T,[V ) 

where jl(y) stands for the number of elements of V and /i*" for the n-times convolution 
of fi. Then the stationary law of X is uniform law on V and that {X,N} is non-strong. 



Proof. Let A denote the stationary law for the process X = {Xk)kez- Since Xq 
NqN_i ■ ■ ■ N^n+i^-n, we have 



A(j) = P(Xo =j)=J2 Pi^oN-i ■ ■ ■ iV_„+i(^) = j)^(X_„ = (1.10) 

iev 

= ^/.*"(aGS:a(z)=j)A(^) (1.11) 



iey 



kE^(^) = i7T7T- (1-12) 



"-°°tl(V^)^ ^^ tl(^) 



Let / be an arbitrary function on V and let / be a negative integer. Since Xq 

NqN^i ■ ■ ■ Ni+iXi, we have 



(1.14) 

(7=NoN-i-Ni+i 



E[fiXo)\a{N, : J > / + 1)] = E[f{aX,)]l^^^^_^,„^^^^ (1.13) 

f{ax)X{dx 

Since Supp(/i) C &(y) and since the uniform law on V is (5(V)-invariant, we see that 

E[f{Xo)\a{Nj : J > / + 1)] = / f{x)X{dx) = E[f{Xo)]. (1.15) 

Jv 

Letting Z — > — oo, we have 

E[/(Xo)|a(iV, : J G Z)] = i5;[/(Xo)]. (1.16) 

This shows that Xq is independent of (y{Nj : j El). D 

1.4 Backgrounds 

Let us give a brief remark on the backgrounds of this study. 

a). Road coloring problem. What we call the road coloring problem is the following: 

Does a directed graph admit at least one synchronizing road coloring? (1-17) 

This problem was first posed by Adler-Goodwyn- Weiss [1] (see also j2]) in the context of 
the isomorphism problem of symbolic dynamics with a common topological entropy. It 
was solved only recently by Trahtman [T3], so that the problem is now a theorem: 

Theorem 1.5 (Trahtman [l3]). A directed graph which satisfies the assumption (A) 
admits at least one synchronizing road coloring. 

See also [B], [S] and |1] for some other developments before Trahtman [13j. 

b). Tsirelson's equation in discrete time. Stochastic equation (II. 6p is related 
to the study [7j by Tsirelson, who introduced, in order to construct an example of a 



stochastic differential equation which has a non-strong solution, a stochastic equation 
indexed by the negative integer 

Xk = Nk + Xk-^, A;G-N, (1.18) 

where X and A^ take values in the one-dimensional torus M/Z. 



Yor [IH] obtained a necessary and sufficient condition in terms of the law of the driving 
noise process for a strong solution of equation fll.lSp to exist. Hirayama-Yano [S] studied 
Tsirelson's equation in discrete time 

Xk = NkXk-u k e -N, (1.19) 

for processes {Xk)k&-'N and {Nk)k£-'N taking values in a compact group and obtained a 
necessary and sufficient condition in terms of infinite product of the driving noise process 
for a strong solution of equation (I1.19P to exist. Hirayama-Yano [lO] studied equation 
(I1.19P for processes taking values in a compact space with semigroup action and obtained 
some sufficient conditions for a strong solution of equation 01.19P to exist and not to exist. 

For other contributions of Tsirelson's equation in discrete time, see Akahori-Uenishi- 
Yano [3] and Takahashi fn\. Several reviews on this topic can be found in ^E\, [U] and 

m- 

c). Finite-state Markov chain. As we shall see later in Theorem 12. 8 [ for a /i-random 
walk {X, N}, the process X is a finite-state Markov chain which is stationary, irreducible 
and aperiodic. Yano-Yasutomi [TB] studied its converse and proved the following: Any 
finite-state Markov chain which is stationary, irreducible and aperiodic can be realized as 
a /i-random walk subject to a synchronizing road coloring. 



1.5 Organization of this paper 

The remainder of this paper is organized as follows. In Section [2], we introduce some more 
notations and discuss existence and uniqueness of yU-random walks. In Section [3|, we give 
several examples which help the reader to understand our main theorems deeply. Section 
m is devoted to the proof of Theorem 11.31 In Section [5l we discuss periodic case. 



2 Notations and preliminary facts 



2.1 Directed graphs and their road colorings 



If the set V consists of m elements, we may and do write V = {1, . . . ,m}. We shall 
identity i E V with Vi where {vi, . . . ,Vm} is the standard basis of M™' defined as 



Vl 



"l" 




"o" 





, V2 = 


1 











(2.1) 



We remark that the product ax has two meanings: one is the image of the site x E V 
by the mapping cr G S, and the other is the usual product among an m x m-matrix 
[o'{y,x)]y^xev and a m- vector (or m x 1-matrix) x in M™. 

The identity mapping e is identified with the identity matrix. We write 
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1 1 






, 02 





1 1 








1 1 



(2.2) 



The set S is a subsemigroup of the semigroup consisting of all m x m matrices, but it is 
not a group, because the elements oi, . . . ,0m do not possess its inverse in E; in fact, we 
have 



OiX 



X E V, i = 1, . . . ,m. 



(2.3) 



Let So be a subset of S. A sequence s = {ap, . . . ,o"i) of Sq is called a word in So, 
and then (s) denotes the product fXp ■ • ■ ai E S. We note that the road coloring So is 
synchronizing if and only if (s) = Oi for some word s in So and some z = 1, . . . ,m. To 
study the non-synchronizing cases, we introduce the following. 

Definition 2.1. Let Sq be a subset of S. Let Vq be a subset of V . 

(i) Vo is called synchronizing if (s) Vq is a singleton for some word s in Sq. 

(ii) Vo is called a deadlock if Vq has no synchronizing pair. 

(iii) Vq is called stable if the subset (s) Vq is synchronizing for all word s in Sq. 



Note that there may exist a pair which is synchronizing but non-stable; an example 
will be given in Section 13.31 

Definition 2.2. A subset Vq of \^ is called an F- clique if Vq is a deadlock and is of the 
form Vq = (s) V for some word s in So. 



We write }](■) for the cardinality and set 

rh = min{tl((s) V) : s is a word in Sq}. (2.4) 

The following lemma can be found in Friedman [8]. 

Lemma 2.3 (Friedman |H]). The following assertions hold: 

(i) IfVo = (s) V for some word s in Sq such that rh = tl((s) V), then Vq is an F-clique. 

(ii) For any F-clique Vq and for all word s in Sq, the subset (s) Vq is also an F-clique 
and satisfies tl(Vo) = tl((s) Vq). 

(iii) Every F-clique has in elements. 



Let us give the proof of Lemma 12.31 for completeness of this paper. 

Proof, (i) Suppose that (s) V had a synchronizing pair. Then there would exist another 
word s' in Sq such that tl((s's) V) < ti((s) V^) = rh. This contradicts the minimality of rh. 
Thus we obtain (i). 

(ii) Let Vq be an F-clique and s be a word in Sq. Suppose (s) Vq were not an F-clique. 
Then it would admit a synchronizing pair, and so would Vq. This is a contradiction. 

Suppose that tl(Vo) > tl((s) Vq). Then Vq would admit a synchronizing pair, which is 
again a contradiction. 

Hence we obtain (ii). 

(iii) Let Vq be an F-clique and s be a word in Eq such that rh = ^{{s)V). Then, by 
(ii), we see that 



M 



m < tl(K)) = Kis) Vo) < tl((s) V) = fh, (2.5) 

which shows that m = tl(Vo). D 

2.2 Random walks 

We deal with a pair of two processes X = {Xk)kez and A^ = {Nk)k&i defined on a common 
probability space. We need the a-fields generated by these processes up to time k E Z 
given by 

J-f = a(X,:j<A;), J-f = a(iV, : j < A;), J-f ^ = a(X,, iV,- : j < A;). (2.6) 

We also need the following a- fields for each k,l E 7^ with k > I: 

J-J = a(X,- : J = fc, A; - 1, . . . , / + 1), F^^, = a{N, : j = k,k - 1, . . . J + 1) (2.7) 

and 

J'kf = ^(Xj, N,:j = k,k-l,...,l + l). (2.8) 



Definition 2.4. Two pairs of processes {X, N} and {X', N'} are called identical in law 
if 

{iXk)kez, {Nk)k&) = {{XDkez, {K)kez) • (2.9) 

In this case, we write {X, N} = {X', N'}. 

Definition 2.5. A pair of processes {X, N} is called stationary if, for any n G Z, {X, N} 

and {{Xk+n)k&, {Nk+n)k&z} are identical in law. 

Lemma 2.6. Let {X, N} and {X', N'} be two pairs of processes. Then the following 
assertions hold: 

(i) // {X, N} = {X', N'} and if {X, N} is a ^-random walk (resp. stationary) , then 
{X', N'} is also a ^-random walk (resp. stationary) . 

(ii) // {X, N} and {X', N'} are fi-random walks, and if X^ = X^ for all k E 2,, then 
{X,X} = {X',X'}. 

Proof. Claim (i) is obvious. Let us prove Claim (ii). Let k^ G Z. Since X^^.i = X'f^^_^ 
and N = N' and since X^ = X^X^.i ■ ■ ■ X^gX^^.i for k > /cq, we see that (X^, Nk)k>kg = 
(X(, N',.)k>ko- Since /cq G Z is arbitrary, we obtain (X^, Nk)k(iZ = {Xj., Nj.)kez- □ 

The convolution of two probability laws /ii and /i2 on E will be denoted by /ii * /i2, 
which is a probability law on S such that 

(/xi * /X2)(ct) = Yl /^i(^i) Y /^2((T2)l{ai<T2=<7}, a G S. (2.10) 

(tiGS CT26S 

For a probability law /i on S and A on y, the convolution of /i and A will also be denoted 
by ;U * A, which is a probability law on V such that 

(/x*A)(i/) = 5^/i(^)E^(^)l{-=^}' y^^- (2.11) 

By (II. 2p . we see that 

(a^i *yU2) * A = /ii * (/i2 * A). (2.12) 

We write fx*^ = /i and define //*" for n > 2 recursively by /i*" = /i**^""^^ * /x. 

Lemma 2.7. Let {X, X} be a ^-random walk. For A; G Z, let A^ denote the law of X^. 
Then the following convolution equation holds: 

Afc = /x*Afc_i, A; G Z. (2.13) 

Conversely, if probability laws /i and {A^ : A; G Z} are given and the convolution equation 
fl2.13p is satisfied, then there exists a ^-random walk {X, X} and X^ has law A^ for all 
A; G Z. The ^-random walk is unique up to identity in law. 



Proof. Let {X, N} be a /i-random walk. Since X^ = NkXk_i and since N^ is independent 
of Xfc_i, we obtain ( 12.1 3p . 

Let yU and {A^ : A; G Z} be given such that fl2.13p holds. Then, by the Kolmogorov 
extension theorem, we may construct a (possibly time-inhomogeneous) Markov chain 
(Xfc, Xk^ijk^i with state space V^ x S so that the marginal law for each /c G Z is given as 

P{Nk = a, Xk-i = x)= fi{a)\k-iix), aeJ:, xeV (2.14) 

and the one-step transition probabilities are given as 

P{Nk = ?, Xk-i = y I Nk-i = a, Xk-2 = x) = /i(?)l{y=^^} (2.15) 

for all cr, <; G S and all x,y eV . It is easy to see that the so constructed pair of processes 
{(Xfc)fcgz, {Nk)kei} is as desired. The uniqueness is immediate from Lemma YIM D 

2.3 Aperiodic case 

Since the index of our /x-random walk varies in Z, existence and uniqueness of /x-random 
walks are not obvious. The following theorem assures the existence and uniqueness in the 
aperiodic case. 

Theorem 2.8. Let fi be a probability law on S and suppose that the directed graph induced 
by fi satisfies the assumption (A) . Then the following assertion holds: 

(i) There exists a fi-random walk {X,N} in V. 

(ii) The ^-random walk is unique up to identity in law. 

(iii) The fi-random walk {X, N} is stationary. 

(iv) The common law X of X = {Xk)kei is a unique probability law satisfying 

/i*A = A. (2.16) 

(v) The tail a -field J^_^ '-^Ok-^k' is a.s. trivial. 

This theorem is an immediate consequence of the classical Perron-Frobenius theory on 
infinite product of stochastic matrices. We call B = {B{y,x))x,y=i,...,m a stochastic matrix 
if B{y, x) > ioT x,y E V and 

^E(y,x) = l, xeV. (2.17) 

yev 

We call a column vector u = ^ [""(1) ■ ■ ■ u{m)'^ a stochastic vector if u{x) > for all 
X = 1, . . . ,m and J2l!^=i ^(^) = 1- 
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Theorem 2.9 (Perron-Frobenius). Let B = (B(y , x))x,y=i,...,m be a stochastic matrix and 
suppose that there exists a positive integer r such that every entry of B^ is positive. Then 
there exists a stochastic vector u such that 

B"iy,x) -^ u{y) for all y = 1, . . . ,m, (2.18) 

where B'"-{y,x) is the {y,x)-entry of B"-, i.e., B" = {B"-{y,x))x,y=i,...,m- The vector u is 
the unique stochastic vector such that 

Bu = u. (2.19) 



For the proof of Theorem 12. 9[ see, e.g., [TT] . 



Let us give the proof of Theorem 12.81 for completeness of this paper. 

Proof of Theoreni \2.S\ . (i) Define a (l^ x '\/)-matrix B = {B{y,x))x.yev by 

5 = 5^/i(a)a, (2.20) 

or in other words, 

B{y, x) = fi{cxe^:ax = y), x,yeV. (2.21) 

Note that i? is a stochastic matrix and that B{y, x) is positive if A{y, x) > 1. Since (V, A) 
satisfies the assumption (A), there exists a positive integer r such that every entry of A^ 
is greater than or equal to 1, and hence that every entry of B^ is positive. Thus we may 
apply Theorem 12.91 to see that there exists a probability law A on V^ such that 

/i™({aGS:ax = i/}) -^ A(y), x,yeV, (2.22) 

n— >oo 

and that A is the unique probablity law such that 

/i*A = A. (2.23) 

By this convolution equation, we may apply Lemma 12.71 to construct {X, N} such that 
N has common law /i and X has common law A. This is as desired. 

(ii) Let {X, N} be a /i-random walk. For each fc G N, let A^ denote the law of Xk- 
Then we have 

Afc = /i*^^-') * A;, kJeZ, k> I. (2.24) 

Let /;; G Z be fixed. Since V is finite, there exist a subsequence l{n) — )■ — oo and a 
probability law A such that Xi(^n) —^ A. Then, for any y G V, we have 

A,(y)=^*(^-'("»*A,(„)(y) (2.25) 

= ^^*(fc-/(n))(|^ G S : aa: = y})A,(„)(x) (2.26) 

xeV 

^X{y)J2x(x) = Xiy). (2.27) 

x&V 
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This shows that X^ has law A for each fc G Z. This proves uniqueness by Lemma 12.61 

Claims (iii) and (iv) have already been proved. 

(v) Suppose that there were A G J'f^ such that < P{A) < 1. Define P' = P(- | A). 
Then it is easy to see that {X, N} under P' is also a /^-random walk. Thus Claim (ii) shows 
that P'{{X,N) G ■) = PiiX,N) e ■)• This contradicts the fact that P'{A) = 1 > P{A). 
This proves Claim (v). 

The proof is now complete. D 

2.4 Proofs of easy parts of Theorem 11.31 

Let us prove easy parts of Theorem 11.31 

Proof of [{i) =^ (ii)] of Theorem \1.3[ Suppose that (i) holds. Let s = (cTp, . . . , cti) be a 
word in Supp(/i) such that (s) V^ is a singleton. Define, for each A; G Z, a random time 

T{k) = max{/ = k - l,k - 2, . . . : Ni+p = ap, . . . , Ni+i = ai}. (2.28) 

Here we note that the random time k — T{k) is a stopping time with respect to the 
filtration {J^kk-n : ^ = 1, 2, . . .}. Then we see that T{k) is finite a.s. and that 

lim Nk,i = Nk,Tik)+p a.s. (2.29) 

/— >— oo 

Thus we obtain (ii). D 

Proof of [{ii) =^ (iii)] of Theorem \l.^ Suppose that (ii) holds. Denote 

Yfc = lim NkNk-i ■ ■ ■ Ni+i a.s. for k e Z. (2.30) 

Then, for any fixed Xq G l^, a pair of processes 

{iYkXo)k&,{Nk)k&} (2.31) 

is a strong /i-random walk which is identical in law to {X, N}. This proves (iii). D 

3 Illustrative examples 

Before proceeding to prove our main theorems, we give illustrative examples. 
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Figure 3. 



Figure 4. 



3.1 Synchronizing case 

Let V = {1, 2, 3} and consider the following adjacency matrix: 

^A(l,l) A(l,2) A{1,3T 
A= A{2,1) A{2,2) A{2,3) 
A(3,l) A(3,2) A(3,3) 





'0 1 1" 


= 


1 1 




1 1 



(3.1) 



We can easily verify that the graph {V,A) satisfies the assumption (A). Consider a road 
coloring {a^^\a'''^^} of {V,A) given as 



a 



(1) 



'0 





1" 











1 


1 






a 



(2) 



"0 


1 


0" 


1 





1 












See Figure 3, where the thick roads are colored a^^^ and the thin ones o"*-^-*. Since 



a(i)cr(2) = 03, 



(3.2) 



(3.3) 



we see that the road coloring {a^^\a^'^^} is synchronizing. 

Let p,q > with p + q = 1 and let // be a probability law on S such that 

f^iW^)=P, KW^'^}) = q. (3.4) 



Let {X, A^} be a /x-random walk in {V,A). Solving equation (I2.16p . we see that the 
common law A of X is given as 



■a(i)- 

A(2) 
A(3) 



2 + pq 



1 — pq 
q + pq 
p + pq 



where we write X{i) simply for X{{i}), i = 1,2, 3. For each A; G Z, we have 

T{k) = inf{n = 1,2,...,: Nk-n+i = ct^'\ Nk-n+2 = cr^'^} 
and then we have 

Xk = Nk,k-T(k)+2Vz, k eZ, 
which shows that the yU-random walk {X, N} is strong. 



(3.5) 

(3.6) 
(3.7) 
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3.2 Non-synchronizing case: an easy example 

Let V = {1,2,3} and let A as defined in f l3.ip . Consider a road coloring {a^^\a'''^^} of 
(y. A) given as 



a 



(1) 



1 




1 


1 


(2) 


1 


1 




1 



(3i 



See Figure 4, where the thick roads are colored a^^^ and the thin ones a^'^\ Since 

^(1)^(2) ^ ^(2)^(1) ^ .^^ ^3 9) 

the set G = {a^^\ a^'^\ id} is a group, and thus we see that the road coloring {a^^\a^'^^} 
is non-synchronizing. 

Let /i be a probability law on E such that Supp(/i) = {a^^\a^'^^}. Let {X,N} be a 
yU-random walk in (V, A). Solving equation (12.161) . we see that the common law A of X is 
uniform on V, i.e., 



(3.10) 



rA(i)i 


1 


"1" 


A(2) 


Q 


1 


La(3)J 


O 


1 



Now, by Proposition II. 4[ we conclude that {X, X} is non-strong. 



3.3 Non-synchronizing case: a difficult example 




Figure 5. 



Let V = {1, 2, 3, 4, 5} and consider the following adjacency matrix: 



A 












1 


0" 


2 








1 








1 

















1 





1 





1 


1 





1 



(3.11) 
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We can easily verify that the graph (V, A) satisfies the assumption (A). Consider a road 
coloring {a^^\a'^'^^} of (V, A) given as 



a 



(1) 












1 


0" 


1 

















1 

















1 




















1 



O" 



(2) 















0" 


1 








1 
































1 





1 


1 









(3.12) 



See Figure 5, where the thick roads are colored a^^^ and the thin ones a^'^\ It is easy to 
see that the F-cliques are 



{1,3,5} and {2,4,5}. 
In particular, the road coloring is non-synchronizing. The pairs 

{1,2},{1,4},{2,3},{3,4} 
are all synchronizing. Since 

{1,4} ^ {1,2} ^ {2,5}, {2,3} ^ {3,4} ^ {2,5}, 

and since {2, 5} is a deadlock, we see that the pairs f l3.14p are non-stable. 



(3.13) 



(3.14) 



(3.15) 



Let p,q > with p + q = 1 and let /i be as defined in (13. 4p . Let {X, N} be a /i-randoni 
walk in {V, A). Solving equation (I2.16p . we see that the common law A of X is given as 



■A(l)" 
A(2) 
A(3) 
A(4) 
A(5) 



3(1 +p) 



P 
1 

P 
1 

l + p 



(3.16) 



Thus we may find the uniformity: 

A({1,2}) = A({3,4}) = A({5}). 



(3.17) 



Note that this is a special case of Theorem 14. II given in the next section. One may expect 
that some symmetry lies behind this uniformity, but it seems hidden because {1,2} and 
{3, 4} cannot be interchanged. We will reveal a certain hidden symmetry behind this 
uniformity in the proof of Claim (II. 8p . 



4 Non-strongness of the /i- random walk in the non-synchronizing 
case 

This section is devoted to the proof of Claim (II. 8p . which will complete the proof of 
Theorem 11.31 
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4.1 Uniformity 

Suppose that {X, A^} and A be as in Theorem 12.81 Let Supp(;u) = {a^^\ . . . , a^'^^} and 
suppose that Supp(/i) is non-synchronizing. Then, by (i) of Lemma 12. 3[ there exists a 
word s = {ap, . . . , cxi) in Supp(/x) such that (s) V is an F-chque. We enumerate (s) V as 

{s)V = {xu...,xa} (4.1) 

where 

rh = min{jj((s) V) : s is a word in Supp(yu)}. (4-2) 

Since Supp(/i) is non-synchronizing, we have 

m > 2. (4.3) 

Set 

Vi = {xeV: {s)x = Xi}, i = l,...,m. (4.4) 

Then the family {Vi, . . . , Vfh} is a partition of the state space V . Note that this partition of 
V may depend on the choice of the word s in Supp(/i) such that {s)V is an F-chque. The 
following theorem is crucial to our proof of Claim (II. 8p , which does not matter whatever 
we choose as such a word s in Supp(yu). 

Theorem 4.1. It holds that 

KVi) = ■■■ = KVfh) = 4. (4.5) 

m 

We shall postpone the proof of Theorem 14.11 until Section 14.31 

4.2 Constructing a permutation process 

Denote V = {1, . . . , fh} and its permutation group by GiV). We decompose V into the 
disjoint union UIli ^ where 

V^. = {x e 1/ : (s) X = Xi}, i e V. (4.6) 

By (ii) and (iii) of Lemma 12. 3^ we see that 

tl((s)a(s)r) =m for any a eS, (4.7) 

and hence 

((s) a-){xi, . . . , Xrh} = {xi, ..., Xff,}. (4.8) 
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This yields that there exists a mapping M : E 9 a i— ?■ M[(j\ G &{y) such that 

M[(j]{i) = J if and only if axi e Vj, (4.9) 

where i,j G V. 

Let {X, N} be a /i-random walk in (V, A). Set Ti = and define Tq, T_i, . . . recursively 
by 

T«; = max{/ < T^+i - p : iV^+p = cXp, . . . , iVz+2 = cr2, Ni+i = ai}. (4.10) 

By the second Borel-Cantelh lemma, we see that the decreasing sequence {T^)Ke-N is 
well-defined a.s. Note that, for any k G — N and any I G — N, we have 

{T« = /} G J-o^,. (4.11) 

Now we define an &{V)-vaAned process (Ni^),^^^-^ as 

N. = M [Nt^ ■ ■ ■ iVr_,+p+2iVT._i+p+i] if T^-i <T,-P (4.12) 

and Nf^ = identity if T^-i = T,^ — p. We may write Ji for the law of Nq on ©(V^). Then it 
is obvious that {Nk,)^^-^ has common law fl since 

Nt.--- iVT_,+p+2iVT„_,+p+i = iVo ■ ■ ■ iVr_,+p+2iVT_,+p+l. (4.13) 

By the aperiodicity assumption, there exists a constant r such that from any x & V 
to any y & V there exists a path of length r. For a technical reason, we introduce the 
following assumption: 

p > r and, for any q=p — l,p — 2,...,p — r, 

{Cp, 0"p--l, • • • , O-p-g+2, CTp-q-f-l) ^ (o-q, CTy-i, . . . , (J2, CTl). 

To prove Theorem 14. H we may assume (I4.14p without loss of generality. For this, it 
suffices to replace s by another word s in Supp(/i) defined as follows. Set p = p + 2r and 

{ai if i = 1,2, ...,p, 
ct(i) iii=p + l,p + 2,...,p + r, (4.15) 

a'^^) if i = p + r + l,p + r + 2, . . . ,p + 2r, 

and then define 

s = (%,... a2,ai). (4.16) 

Then it is obvious that the sequence 's satisfies 04.141) . By (ii) and (iii) of Lemma 12.31 we 
see that (s) V is also an F-clique, and that (s)V = {xi, . . . ,Xffi}. Set 

Vi = {x & V : (s) X = Xi}, i = 1, . . . ,m. (4-17) 

We then note that (Vi, . . . , Vrh) is a permutation of (V^i, . . . , Vfh), which shows that the 
replacement of s by s^ does not matter in the proof of Theorem 14. 1[ 
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Lemma 4.2. Suppose that fl4.14p holds. Then, for any i,j G V, it holds that 

/2™ ({tt G &{V) : 7r(0 = j|) ^ 4. (4.18) 

Proof. By the strong-connectedness property of (V, A) we see that, for any i,j G V^, there 
exists a word s' = (a^, . . . , aj) in Supp(/i) such that 

{s')%eVj. (4.19) 

We define an event B by 



N-r = ap,. . . , N^r-p+2 = 0-2, iV-r-p+1 = O"!}- 



(4.20) 



Since we assume that (14.141) holds, we see that 

T_i = — r — p and A^o(0 — J o'^ ^^^ event S. (4.21) 

Therefore, for any i, j G V", we have 

/2({7r G ©(t/) : 7r(z) = ]}) > P{B) > 0. (4.22) 

Now we may apply Theorem 12.91 to see that there exists a probability law (p(j) : j ^ V) 
such that 

/I*"({7r G ©(i/) : 7r(0 = j}) ^ p(j), ^,J G t/. (4.23) 

Since, for any n and j G l^, we have 

5^/2™({7r G 6(i/) : vr(z) = j}) = $^/2™({7r G ©(\/) : 7r(j) = z}) = 1, (4.24) 

we see that p{j) = 1/rh for all j G V^. The proof is now complete. D 

4.3 Constructing a new random walk 

Define X^ G V^ as follows: 

X^ = i if Xt, G \>i. (4.25) 

We will prove gradually that the process {X,X} = {(XK)Kg_N, (Xk)kg-n} is a /i-random 
walk indexed by — N. The first step is the following. 
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Lemma 4.3. Suppose that f l4.14p holds. Then, for fixed k G — N, the following assertions 
hold: 

(i) Xk = Ni^Xf^^i holds a.s.; 

(a) Xk-i is indepenent of N^. 

(ill) P(X^_i = i) = XiVi) for all i G V. 

Proof. Claim (i) is obvious by definition. Let us prove (ii) and (iii) at the same time. Let 
/ G — N, TT G &{V) and i &V . Then we have 

P (n, = TT, X,_i = i, T«„i = /) 

=P (iV, = n, Xie V„ T,„i = /' 
=P (iV, = TT, T,_i = /) P{Xi = i) 
=P (iV, = TT, r,_i = /) X{Vi) 

Summing up by Z G — N, we obtain 





(4.27) 


(by independence) 


(4.28) 


(by stationarity). 


(4.29) 



P (iV. = vr, X,_i = i^=p(^N, = vr) A(V;), 



which proves Claims (ii) and (iii). 

The second step is to prove Theorem 14.11 



(4.30) 



D 



Proof of Theorem \4.1\ Define a probability law A on V by 

A(i) = X{Vi) for all i G V. 
By Lemma [4. 3t we have 

A = /i * A. 
Iterating this convolution equation, we obtain, for any fixed i & V, 

A(H)=(/2*-*A)(«) 

= 5^/2*"({7rG6(V):7r(j) = 0)A({j}) 



n~>-oom ^-^ m 



j&V 



where we have used Lemma [4.21 This completes the proof. 



(4.31) 
(4.32) 

(4.33) 
(4.34) 

(4.35) 

D 
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The third step is the following, which reveals a symmetry hidden behind the uniformity 
(14. 5p in Theorem 14.11 

Theorem 4.4. Suppose that (14.141) holds. Then {{Xf^)i^^_^^ (^K)Ke-N} ^s a Jl-random 
walk indexed by —N, i.e., 

(i) Ni^ is independent of T^li for all k, G — N; 

(a) (A^k)kg-n is IID with common law yU; 

(Hi) Xk = XkXk-1 holds a.s. for all k, G — N. 

Moreover, it holds that 

(iv) X^ has uniform law on V for all k G — N. 

Proof. We have already shown (ii), (iii) and (iv). Let us prove (i). For this, it suffices to 
prove that, for any fixed k G — N, 

Nq, N_i, . . . , Xk and X^-i are independent. (4.36) 

Let B^ G a{No,N^i,...,N^) and i e V. Let / G -N. Since B^ n {T«,_i = /} G 
(t{No,. . .,Ni+i), we have 



P ^B,, X,_i = t, r,_i = I) (4.37) 

=P {b^, Xi G i>„ T«_i = /) (4.38) 

=P (5«, T^_i = /) P te G V^ (by independence) (4.39) 

=—P(B^,T^_i = l) (by Theorem El). (4.40) 

m 

This proves that X^-i is independent of o"(Xo, X_i, . . . , X^). 

Let 5«+i G (t(Xo,^-i,---,^k+i) and vr G ©(r). Let /,/' G -N with / > /'. Since 
B^+i n {T« = /} G a(Xo, . . . , X^+i), we obtain 



P i^B^+u N, = vr, T, = I, T,_i = I'j (4.41) 

=P (P«+1, T^ = l, M [NiNi^i ■ ■ ■ Xz,+i] = TT, T_i o 01 = r) (4.42) 

=P (P.+i, T, = /) P (m [X,X,_i ■ ■ ■ X,+i] = TT, T_i o e, = r) (4.43) 

=P (P.+i, T, = /) P (m [XoX_i ■ ■ ■ X,_i+i] = TT, T_i = r - /) (4.44) 

=P (P.+i, T, = /) P (Xo = TT, r_i = /'-/), (4.45) 

where we write 

T_i 06*; = max{A; < -p : X^+^+p = (Tp, . . . , X^+fc+i = ai}. (4.46) 
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Summing up (H^TO - (10S1) by /' G -N, we have 

P (s,+i, iV, = vr, T, = /) = P (5«+i, T^ = l)P (iVo = vr) . (4.47) 

This shows that A^^ is independent of cr(A^O; • • • ; -^k+i)- Therefore, we conclude that 
A^o, A^-i, • • • , A'k+i and A^k are independent, which completes the proof. D 

For k G — N, we write 

K{k) = max{K E -N:k-p> T«}. (4.48) 

Note that 

L{k) := TK{k) = max{/ < k - p : Ni+p = ap, . . . , N1+2 = 0-2, A^/+i = ai} (4.49) 

and that 

{Lik) = l}ea{Nk,Nk-i,...,Ni+i), l<k-p. (4.50) 

The following theorem proves Claim (II. 8p . 
Theorem 4.5. Suppose that (I4.14p holds. Then, for any k G — N, it holds that 

Xk e T^ V aiXKik)) a.s. (4.51) 

and that 

XK(k) is independent of J-'q and has uniform law on V. (4-52) 

Consequently, if the road coloring is non-synchronizing, i.e., m > 2, the ^-random walk 
{X, A^} is non-strong. 

Proof. By definitions of L{k) and K{k), we have 

Xk =NkNk-i ■ ■ ■ NL{k)+p+iNL{k)+p ■ ■ ■ NL{k)+iXL{k) (4.53) 

=NkNk-i ■ ■ ■ Ari(,)+p+ia("(i)) ■ ■ ■ a("(P))Xz.(fe) (4.54) 

=NkNk-i ■ ■ ■ N^k)+p+iXi (4.55) 

with i = XK{k)- Since NkNk-i ■ ■ ■ NL{k)+p+i G ^k 1 ^'^ obtain (I4.5ip . 

Let k' < k and let B G a{Nj : j >k' + 1). Let I < k - p and l' < mm{k' - p, I}. Note 
that we have 



P l^XLik) e V„ L{k) = I, L{k') = I', Bj (4.56) 

-P (Xi G % L{k) = I, L{k') = I', fi) (4.57) 

-P (Xr ■ ■ Ni,+p+ia+X, G V,, L{k) = /, L{k') = I', p) (4.58) 

fh 

= J]P(X, gV„ M[X,---X,+p+i](j)=^, L{k)=l, L{k') = l', By (4.59) 



j=i 
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Since X// is independent of Tq^i := o"(A^o, • • • , ^f+i), we see that 



-^o^/'j = P [Xv e V,) = ^. (4.60) 

Since {L{k) = I, L{k') = I'} G J^^ii, we obtain 

(I139D=^5^P(M[iV,...iVp+p+i](j) = ^, i:(A;) = /, L{k') = l\ b) (4.61) 

"^ i=i 

^P fL(A;) = /, Lik') = I', b) . (4.62) 

n \ / 



m 
Summing up in / < A; — p and V <k' — p, we have 



P (Xkw = 2, p) = P (Xi(fc) e V^„ P) = irP{B). (4.63) 

This proves (gSS)- □ 



5 Periodic case 

If a directed graph (V, A) is strongly-connected, then it is easy to see that the greatest 
common divisor among {n > 1 : A'^{x,x) > 1} does not depend on x G V^, so that it is 
called the period of (V, A). If the period of (V, A) is greater than one, then (V, A) is called 
periodic. 

Let us study periodic case. We shall utilize the following theorem. 

Theorem 5.1 (Perron-Frobenius). Let fi be a probability law on S. Suppose that the 
directed graph induced by /i is strongly- connected and has period d > 2. Then there exist a 
partition {V^^\ . . . , V^^''^} ofV and a family {X^^\ . . . , A^*^^} of probability laws on V such 
that the following assertions hold: 

(i) for each x G V^^\ it holds that 

fi{a e i: : y = ax) < . (5.1) 

1 = otherwise, 

where V^'^+^'^ = V^^\ 
(ii) for each i = 1, . . . ,d, the support of A^'^ is V^^\- 
(iii) for each i = 1, . . . ,d and each x,y G V^^\ it holds that 

r\oe^:y = ax) ^ A«(y). (5.2) 

(iv) for each i = 1,. . . ,d, it holds that fi*'^ * A^*) = A^*^ 



22 



For the proof of Theorem 15 -H see, e.g., flT]. Each V^^'' will be called a cyclic part. 

In the sequel, let /i, {V^^\ . . . , V^'^^ and {A^^), . . . , A^'^)} as in Theorem O The fol- 
lowing theorem chracterizes the class of all /i-random walks. 

Theorem 5.2. The following assertions hold: 

(i) For each i = l,...,d, there exists a fi-random walk {X^^\ N^^^} in V such that 
Xq G V^^^ a.s. Such a fj-random walk is unique up to identity in law and the tail 
a-field J-'_oo ' is trivial. 

(ii) Let {X, N} be an arbitrary ^-random walk in V. Then, for each i = 1, . . . ,d, the 
following hold: 



X,N _ 
oo > 



(ii-1) the event A^"^^ = {Xq E V^^^ belongs to the tail a-field T\ 

(ii-2) z/P(AW) > 0, then {X,N} under P{- \ A^) is identical m law to {X«,A^W}; 
in other words, it holds that 

d 
P{{X,N) G-) = $^i'((^^*\Ar») G-)P(A«). (5.3) 

j=i 

Proof, (i) For k = nd + r with n E "L and r = 1,2, . . . ,d, we define A^ = /i**" * A'-*-'. By 
(iv) of Theorem 15. 1[ we may apply Lemma [2.71 to construct a //-random walk {X^^\ N^^^} 
in V such that X], has law A^ for all k E Z. By (ii) of Theorem 15.11 we have Xq G V^^^ 
a.s. 

Conversely, let {X^^\ N^^^} be a /i-random walk in V such that N^'^^ has common law 
/i and Xq G V^^'^ a.s. For k E'L, let Afc denote the law of X^ . By (i) of Theorem 15. 1[ we 
see that \-nd should have its support contained in V^'^'^ for all n E'H. By (iii) of Theorem 
15. 1[ we see that 

Ao = //*""* A_„, — > A». (5.4) 



71— >00 



This proves uniqueness by Lemma [2.71 The tail triviality can be proved in the same way 
as in the proof of Claim (v) of Theorem 12. 8[ 

(ii) Claim (ii-1) is obvious by (ii) of Theorem 15. II and by the fact that X^ G V'^'^' if and 
only if Xk-d ^ V^'^\ Claim (ii-2) is obvious by the uniqueness in Claim (i). The proof is 
now complete. D 

For i = 1, . . . , (i, we write S'-*-' for the set of all mappings from V^"^' to itself, and define 

Then, by (i) of Theorem 15 . H we see that /i*^*-* is a probability law on S*^*). The following 
theorem gives a necessary and sufficient condition for strongness of /i-random walks. 
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Theorem 5.3. Let {X, N} be a ^-random walk in V . 

(A) // the tail a-field Tj^ is trivial, then the ^-random walk {X, A^} is non-strong. 

(B) Suppose that the tail a-field J-'_^ is non-trivial. Take i = l,...,d such that 

{X,N} = {X^^\N^^^}, which is possible by Theorem \5.^A Then following three as- 
sertions are equivalent: 

(i) Supp(/i*^*'') is synchronizing. 

(ii) The limit liin NkNk-i ■ ■ ■ Ni+i exists a.s. for all fc G Z. 

l—^—oo 

(iii) The fi-random walk {X, N} is strong. 

The proof of Theorem 15.31 is an immediate consequence of Theorem 11.3^ so that we 
omit it. 

Acknowledgements: The author thanks Kenji Yasutomi for his suggestions about 
Lemma 12.31 and the example in Section 13. 3[ The author also thanks Professors Yusuke 
Higuchi and Tomoyuki Shirai for fruitful discussions. 
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